{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# 第三章.神经网络\n",
    "<p>神经网络的一个重要性质是它可以自动地从数据中学习到合适地权重参数.</p>\n",
    "\n",
    "## 3.1.从感知机到神经网络\n",
    "<P>神经网络的例子:我们把最左边的一列称为输入层,最右边的一列称为输出层,中间的一列称为中间层(隐藏层)，事实上,多层感知机构成了神经元</P>\n",
    "\n",
    "## 3.2.激活函数\n",
    "<p>若激活函数以阈值为界，一旦输入超过阈值，就切换输出，这样的函数称为阶跃函数，也就是说感知机中使用阶跃函数作为激活函数</p>\n",
    "<p>如果将激活函数从阶跃函数换成其他函数,就可以进入神经网络的世界了</p>\n",
    "\n",
    "### 3.2.1.sigmoid函数\n",
    "<p>神经网络中使用sigmoid函数作为激活函数,进行信号的转化,转化后的信号被传送给下一个神经元</p>\n",
    "<p>事实上,感知机和神经网络的主要区别就在于前者使用阶跃函数，后者使用其他类型的激活函数，下面我们来观察下两类函数的区别</p>\n",
    "\n",
    "### 3.2.3.阶跃函数的实现\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "outputs": [],
   "source": [
    "def step_function(x): # x为numpy类型数组\n",
    "    y = x>0\n",
    "    return y.astype(int)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-25T07:33:28.566831400Z",
     "start_time": "2023-09-25T07:33:28.556830100Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 3.2.3.阶跃函数的图形"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAARbUlEQVR4nO3df4wc513H8c/Hexf6MyTgo6Q+G1vIpbUggXK4kSqUQGhrp6EWEn8kgQZCK8tSjFKJihgq6B/9C0VAVMWtsSIrFAoWUgM1lYtJJSB/VEF2QpLWCQ6HS+OLA7nQqkVJhW9mvvyxe5flPDO7tnd37pl7vyQrNzvjve8qz370+LvPM+uIEAAgfRuaLgAAMBoEOgC0BIEOAC1BoANASxDoANASU0394o0bN8bWrVub+vUAkKQnnnjilYiYKTvXWKBv3bpVp06daurXA0CSbH+z6hwtFwBoCQIdAFqCQAeAliDQAaAlCHQAaAkCHQBagkAHgJYg0AGgJQh0AGgJAh0AWoJAB4CWINABoCUIdABoiYGBbvuI7Zdtf73ivG1/2va87Wdsv3v0ZQIABhlmhv6wpF0153dL2t77s1fSZ6+8LADApRp4P/SIeMz21ppL9kj6XESEpMdtX2P7uoh4aUQ1Ao363oVcT77wbRURTZeClpi99k3atvHNI3/eUXzBxSZJ5/qOF3qPXRTotveqO4vXli1bRvCrgfH7k8f+XQ985d+aLgMtsu+mH9WB3e8c+fOOItBd8ljpVCYiDks6LElzc3NMd5CE734v0xunO/qzj+xsuhS0xNuufsNYnncUgb4gaXPf8ayk8yN4XmBNyItC3ze9QXNbf6DpUoBao1i2eEzSXb3VLjdK+g79c7TJUhGa2lD2D1FgbRk4Q7f9l5JulrTR9oKkT0qalqSIOCTpuKRbJc1Lek3S3eMqFmhCnoemNrBlA2vfMKtc7hhwPiTdM7KKgDVmqSjUYYaOBDDtAAbIi9BUh0DH2kegAwNk9NCRCAIdGCDLC3roSAKjFBggL4IeOpJAoAMDZEVomh46EkCgAwNkOTN0pIFABwbICnroSAOjFBiAZYtIBYEODLBEywWJINCBAXLWoSMRBDowQFaEpjq8VbD2MUqBAbobi5ihY+0j0IEB2FiEVBDowADdjUW8VbD2MUqBAbKc2+ciDQQ6MAB3W0QqCHRgADYWIRUEOjDAErfPRSIYpcAArHJBKgh0YICMlgsSQaADA/ChKFJBoAM1IqLXcuGtgrWPUQrUyIuQJE0zQ0cCCHSgRtYL9A49dCSAQAdqLAc6PXSkgEAHauT5cqDzVsHaxygFaiwVhSSxbBFJINCBGnnBDB3pGGqU2t5l+4ztedsHSs5/v+2/tf207dO27x59qcDkLeW9GTo9dCRgYKDb7kg6KGm3pB2S7rC9Y9Vl90h6NiJukHSzpD+0fdWIawUmbnmGztZ/pGCYGfpOSfMRcTYiLkg6KmnPqmtC0lttW9JbJH1LUjbSSoEGrKxyoYeOBAwT6Jsknes7Xug91u9BSe+SdF7S1yTdGxHF6ieyvdf2KdunFhcXL7NkYHIyVrkgIcOM0rKpSaw6/oCkpyS9XdJPSnrQ9tUX/aWIwxExFxFzMzMzl1wsMGlZb5ULLRekYJhAX5C0ue94Vt2ZeL+7JT0SXfOSviHpnaMpEWjOytZ/Wi5IwDCBflLSdtvbeh903i7p2KprXpB0iyTZfpukH5N0dpSFAk1YyvlQFOmYGnRBRGS290s6Iakj6UhEnLa9r3f+kKRPSXrY9tfUbdHcFxGvjLFuYCJYh46UDAx0SYqI45KOr3rsUN/P5yW9f7SlAc3L2CmKhDDtAGq8vsqFQMfaR6ADNdhYhJQQ6ECNbGWVC28VrH2MUqBGlrMOHekg0IEafMEFUkKgAzVWli3SckECGKVADW6fi5QQ6EANVrkgJQQ6UIPb5yIlBDpQI1tpufBWwdrHKAVqZLRckBACHajB7XOREgIdqMEMHSkh0IEafAUdUsIoBWrkRSGbGTrSQKADNZaKYFMRkkGgAzXyIpidIxkEOlAjy0PT9M+RCEYqUCMrCnVYsohEEOhAjawIVrggGYxUoEaWF3woimQQ6ECNjA9FkRACHaiRF8G2fySDQAdqZDkzdKSDQAdqZEXBh6JIBiMVqJEXwZdbIBkEOlBjKWfrP9IxVKDb3mX7jO152wcqrrnZ9lO2T9v+p9GWCTSDrf9IydSgC2x3JB2U9D5JC5JO2j4WEc/2XXONpM9I2hURL9j+oXEVDExSVhSa6vAPWaRhmJG6U9J8RJyNiAuSjkras+qaOyU9EhEvSFJEvDzaMoFmZLRckJBhAn2TpHN9xwu9x/q9Q9K1tv/R9hO27yp7Itt7bZ+yfWpxcfHyKgYmiI1FSMkwgV42mmPV8ZSkn5b0QUkfkPR7tt9x0V+KOBwRcxExNzMzc8nFApPW3VhEywVpGNhDV3dGvrnveFbS+ZJrXomIVyW9avsxSTdIen4kVQINWcoLZuhIxjBTj5OSttveZvsqSbdLOrbqmi9K+lnbU7bfJOk9kp4bbanA5OV8YxESMnCGHhGZ7f2STkjqSDoSEadt7+udPxQRz9n+O0nPSCokPRQRXx9n4cAkdDcW0XJBGoZpuSgijks6vuqxQ6uO75d0/+hKA5q3VHD7XKSDqQdQI+fmXEgIgQ7UyLh9LhJCoAM1WIeOlBDoQI3uV9DxNkEaGKlADZYtIiUEOlBjqQh16KEjEQQ6UIMZOlJCoAMVIqIX6LxNkAZGKlAhK7r3oGOGjlQQ6ECFvBfo9NCRCgIdqLA8Q5+m5YJEMFKBClleSBIbi5AMAh2osNJDp+WCRBDoQIV85UNR3iZIAyMVqLDUa7mwygWpINCBCjktFySGQAcqLOW9ZYvM0JEIAh2oQA8dqWGkAhWyotdDp+WCRBDoQIUsZ+s/0kKgAxWW16HTQ0cqCHSgwnIPfbrD2wRpYKQCFdj6j9QQ6EAFbp+L1BDoQIXXNxbxNkEaGKlABbb+IzUEOlAhZ5ULEjNUoNveZfuM7XnbB2qu+xnbue1fHl2JQDNWvuCCjUVIxMBAt92RdFDSbkk7JN1he0fFdX8g6cSoiwSasLxTtMPWfyRimJG6U9J8RJyNiAuSjkraU3Ldb0r6gqSXR1gf0Bh2iiI1wwT6Jknn+o4Xeo+tsL1J0i9JOlT3RLb32j5l+9Ti4uKl1gpMFLfPRWqGCfSy0Ryrjh+QdF9E5HVPFBGHI2IuIuZmZmaGrRFoxBIfiiIxU0NcsyBpc9/xrKTzq66Zk3TUtiRtlHSr7Swi/mYkVQINyFeWLdJDRxqGCfSTkrbb3ibpRUm3S7qz/4KI2Lb8s+2HJX2JMEfq+JJopGZgoEdEZnu/uqtXOpKORMRp2/t652v75kCq2PqP1AwzQ1dEHJd0fNVjpUEeEb9+5WUBzWNjEVJDcxCosLxscZoeOhLBSAUqZEUhW9rADB2JINCBClkR9M+RFAIdqJAXwZJFJIXRClRYygtm6EgKgQ5UyItQhzXoSAiBDlTIaLkgMYxWoEJGywWJIdCBClkRbPtHUgh0oEKWs2wRaSHQgQp5EWz7R1IIdKBCVhSa7vAWQToYrUCFLGeGjrQQ6EAFtv4jNQQ6UCEvQlO0XJAQRitQYSkvaLkgKQQ6UCGn5YLEEOhAhYyWCxLDaAUqZAVb/5EWAh2owLJFpIZAByrkRWiae7kgIQQ6UCErQh1un4uEMFqBCvTQkRoCHaiQc7dFJIZAByoscT90JIZABypw+1ykhkAHKnS/go63CNIx1Gi1vcv2Gdvztg+UnP8V28/0/nzV9g2jLxWYLO62iNQMDHTbHUkHJe2WtEPSHbZ3rLrsG5JuiojrJX1K0uFRFwpMWlaEOvTQkZBhZug7Jc1HxNmIuCDpqKQ9/RdExFcj4tu9w8clzY62TGDy8iI0TcsFCRlmtG6SdK7veKH3WJWPSPpy2Qnbe22fsn1qcXFx+CqBCYsIPhRFcoYJ9LIRHaUX2j+nbqDfV3Y+Ig5HxFxEzM3MzAxfJTBhWdEd4vTQkZKpIa5ZkLS573hW0vnVF9m+XtJDknZHxH+PpjygGflyoHP7XCRkmNF6UtJ229tsXyXpdknH+i+wvUXSI5I+HBHPj75MYLKW8kISM3SkZeAMPSIy2/slnZDUkXQkIk7b3tc7f0jS70v6QUmfsS1JWUTMja9sYLyWZ+j00JGSYVouiojjko6veuxQ388flfTR0ZYGNGe5h87tc5ESGoRAiSxfnqHzFkE6GK1Aiazo9dCZoSMhBDpQYnmGzoeiSAmBDpTI+FAUCSLQgRL5yoeivEWQDkYrUGJ5HTozdKSEQAdK5Gz9R4IIdKBExtZ/JIjRCpTI2PqPBBHoQAm2/iNFBDpQgq3/SBGBDpRY3inK1n+khNEKlGCnKFJEoAMlXv+CCwId6SDQgRJLrENHggh0oERODx0JYrQCJeihI0UEOlAio4eOBBHoQAlun4sUEehAiby39X+aHjoSwmgFSqzM0Gm5ICEEOlAiY9kiEkSgAyVevx86bxGkg9EKlFji9rlIEIEOlMiLkC1tINCREAIdKJEVwQoXJIcRC5TI8oI16EgOgQ6UyIqgf47kDBXotnfZPmN73vaBkvO2/ene+Wdsv3v0pQKTkxfBtn8kZ2rQBbY7kg5Kep+kBUknbR+LiGf7LtstaXvvz3skfbb335G7kBV67UI2jqcGVrz6vzl3WkRyBga6pJ2S5iPirCTZPippj6T+QN8j6XMREZIet32N7esi4qVRF/zos/+le/7iyVE/LXCR2Wvf2HQJwCUZJtA3STrXd7ygi2ffZddskvT/At32Xkl7JWnLli2XWqskacfbr9Ynf3HHZf1d4FLsuO7qpksALskwgV7WSIzLuEYRcVjSYUmam5u76Pwwtm18s7Zt3HY5fxUAWm2YJuGCpM19x7OSzl/GNQCAMRom0E9K2m57m+2rJN0u6diqa45Juqu32uVGSd8ZR/8cAFBtYMslIjLb+yWdkNSRdCQiTtve1zt/SNJxSbdKmpf0mqS7x1cyAKDMMD10RcRxdUO7/7FDfT+HpHtGWxoA4FKw0BYAWoJAB4CWINABoCUIdABoCQIdAFqCQAeAliDQAaAlCHQAaAkCHQBagkAHgJYg0AGgJQh0AGgJd++r1cAvthclfbORX35lNkp6pekiGrAeX/d6fM3S+nzdKb3mH4mImbITjQV6qmyfioi5puuYtPX4utfja5bW5+tuy2um5QIALUGgA0BLEOiX7nDTBTRkPb7u9fiapfX5ulvxmumhA0BLMEMHgJYg0AGgJQj0K2D747bD9samaxk32/fb/lfbz9j+a9vXNF3TONneZfuM7XnbB5quZ9xsb7b9D7afs33a9r1N1zQptju2/8X2l5qu5UoR6JfJ9mZJ75P0QtO1TMijkn48Iq6X9Lyk32m4nrGx3ZF0UNJuSTsk3WF7R7NVjV0m6bci4l2SbpR0zzp4zcvulfRc00WMAoF++f5Y0m9LWhefKkfE30dE1jt8XNJsk/WM2U5J8xFxNiIuSDoqaU/DNY1VRLwUEU/2fv4fdQNuU7NVjZ/tWUkflPRQ07WMAoF+GWx/SNKLEfF007U05DckfbnpIsZok6RzfccLWgfhtsz2Vkk/Jemfm61kIh5Qd2JWNF3IKEw1XcBaZfsrkn645NQnJP2upPdPtqLxq3vNEfHF3jWfUPef55+fZG0T5pLH1sW/xGy/RdIXJH0sIr7bdD3jZPs2SS9HxBO2b266nlEg0CtExC+UPW77JyRtk/S0banbenjS9s6I+M8JljhyVa95me1fk3SbpFui3RsYFiRt7juelXS+oVomxva0umH++Yh4pOl6JuC9kj5k+1ZJb5B0te0/j4hfbbiuy8bGoitk+z8kzUVEKndquyy2d0n6I0k3RcRi0/WMk+0pdT/4vUXSi5JOSrozIk43WtgYuTs7+VNJ34qIjzVdz6T1Zugfj4jbmq7lStBDx7AelPRWSY/afsr2oaYLGpfeh7/7JZ1Q98PBv2pzmPe8V9KHJf187//vU72ZKxLCDB0AWoIZOgC0BIEOAC1BoANASxDoANASBDoAtASBDgAtQaADQEv8H3KLPY8+S91KAAAAAElFTkSuQmCC\n"
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "def step_function(x):\n",
    "    return np.array(x>0,dtype=np.int32)\n",
    "\n",
    "x = np.arange(-5.0,5.0,0.1)\n",
    "y = step_function(x)\n",
    "plt.plot(x,y)\n",
    "plt.ylim(-0.1,1.1) # 指定y轴的范围\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-25T07:38:41.142004100Z",
     "start_time": "2023-09-25T07:38:39.101727600Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 3.2.4. sigmoid的函数实现"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAdbklEQVR4nO3de3TU9Z3/8eebXAgkEG4JIEkAA8hFQCFcVFqvWIRW1ksVVKzVFangsUr7q7Vurdujdtce63ZRKbWu5aKAFgUtLqW7utpaLQkENCA0BCUhSMItITdymc/vj0QMGMkAM/lmvvN6nJMD3/l+MvNyTF585vOZiznnEBGRyNfB6wAiIhIaKnQREZ9QoYuI+IQKXUTEJ1ToIiI+EevVDffq1csNGDDAq5sXEYlIOTk5+51zKS2d86zQBwwYQHZ2tlc3LyISkczs0686pyUXERGfUKGLiPiECl1ExCdU6CIiPqFCFxHxCRW6iIhPqNBFRHxChS4i4hMqdBERn1Chi4j4hApdRMQnVOgiIj6hQhcR8YlWC93MnjezEjP76CvOm5n92szyzWyLmY0JfUwREWlNMDP0F4ApJzl/FTC46Ws28OyZxxIRkVPVaqE7594BDp5kyHRgsWv0PtDNzPqGKqCIiAQnFGvo/YDCZsdFTZd9iZnNNrNsM8suLS0NwU2LiMjnQlHo1sJlrqWBzrlFzrks51xWSkqLn6AkIiKnKRSFXgSkNztOA4pDcL0iIr7TEHBU1zaE5bpD8Zmia4B5ZrYcmACUOef2huB6RUQiVnVtAwX7K8gvqWBnaSU7SyvYWVJBwf5K5lycyf2Th4T8NlstdDN7CbgE6GVmRcDDQByAc24hsBaYCuQDVcB3Q55SRKQdcs5xoLKWnSUV5JdWsLOksbjzSyrYc7j62LgOBhk9OpOZksTXh6Qw8eweYcnTaqE752a2ct4Bc0OWSESknWkIOAoPVjXOskuPn3Ufrqo7Nq5TXAxnpySSNaA7N6akk5mSxKDUJPr37ExCXEzYc4ZiyUVExBeqauspaLY8srO0kvySCnbtr6S2IXBsXK+kjmSmJDJtZF8yU5LITG0s7r5dE+jQoaXnibQNFbqIRBXnHPsrapvNtJtm2y0sk/TvmUhmSiKXnJNCZmpS44w7JYnkznEe/hd8NRW6iPjejn1HeDm7kJxPD7GztJKy6uOXSTJTExk3oDszUtKPzbb79+xMx9jwL5OEkgpdRHypvKaONzbvZUV2IZsLDxMXY4zJ6M43R/Vl0Oez7dQk+ni8TBJKKnQR8Q3nHB/sOsjK7ELWfriXmroA5/Tuwr98czj/dN5Z9Ezq6HXEsFKhi0jE+6yshj9sLGJldiGfHqiiS8dYrhuTxg1Z6YxKS8bMHzPw1qjQRSQi1dYH+J9t+1iZXcj/7Sgl4GDi2T34/hWDmTKiL53iI2v9OxRU6CISUbZ/doSV2YW8umkPBytr6dM1gbsvGcS3s9Lo3zPR63ieUqGLSLtXXlPH65uLWZlddGyDc/Lw3nw7K52vD04hxiebmmdKhS4i7dKxDc4Nhaz9KPo2OE+HCl1E2pXPymp4JaeQl3OKonqD83So0EXEc59vcK7ILuQdbXCeNhW6iHhm+2dHWLGhkNdytcEZCip0EWlTxzY4NxSyuahMG5whpEIXkbBzzvF+wUFeztYGZzip0EUkrFbn7uHJ9Tu0wdkGVOgiEhZ1DQEeW7uN//rrJ4xOS+beG0Zz1bna4AwnFbqIhNyBiqPMfXEj7xcc5PaLBvLg1KHExoTiM+nlZFToIhJSH+0p464lOeyvOMqTN4zm2jFpXkeKGip0EQmZ1zbt4Ud/2ELPxHhemXMhI9OSvY4UVVToInLG6hsCPP7mx/zuL7uYMLAHT988hl565kqbU6GLyBk5WFnLvBc38t7OA9x24QB+Mm0YcVov94QKXUROW15xGbMX51BacZQnrh/Ft7PSvY4U1VToInJaVuc2rpd37xzPy3ddwOj0bl5HinoqdBE5JfUNAf593XYWvVPA+AGN6+UpXbRe3h6o0EUkaIerarnnpU28+4/93HpBfx6aNpz4WK2XtxcqdBEJyra95cxeks2+sqP8+3WjuGGc1svbGxW6iLTqjS3F/PDlLXTtFMuKuyZyfkZ3ryNJC4J6rGRmU8xsu5nlm9kDLZxPNrPXzWyzmeWZ2XdDH1VE2lpDwPGLNz9m3oubGH5WV16/Z5LKvB1rdYZuZjHA08BkoAjYYGZrnHNbmw2bC2x1zn3LzFKA7Wa2zDlXG5bUIhJ2ZVV13LN8E+/sKOXmCRk8/K0RWi9v54JZchkP5DvnCgDMbDkwHWhe6A7oYo3vhZkEHATqQ5xVRNrI9s+OMHtJNsWHq3n82pHMHJ/hdSQJQjCF3g8obHZcBEw4YcwCYA1QDHQBbnTOBU68IjObDcwGyMjQD4hIe/Tmh3uZ//JmEjvGsnz2RMb27+F1JAlSMI+fWnoHenfC8TeAXOAs4DxggZl1/dI3ObfIOZflnMtKSUk55bAiEj4NAccv123ne8s2ck6fLrxxzySVeYQJZoZeBDR/flIajTPx5r4L/MI554B8M9sFDAX+HpKUIhJWZdV1fH/5Jt7aXsqMcek8Mn0EHWP1QRSRJphC3wAMNrOBwB5gBnDTCWN2A5cD75pZb+AcoCCUQUUkPP6x7wizl+RQdKiKR685l5sn9Pc6kpymVgvdOVdvZvOAdUAM8LxzLs/M5jSdXwj8HHjBzD6kcYnmR865/WHMLSIhsC7vM+5fkUun+FheunMiWQO0xBLJgnphkXNuLbD2hMsWNvt7MXBlaKOJSLgEAo6n/ryDX/9vPqPTu/GbW8bSJznB61hyhvRKUZEoU15Tx33Lc/mfj0u4ISuNf51+LglxWi/3AxW6SBTJL6lg9pJsdh+o4l+nj2DWxP40vnxE/ECFLhIl1m/dx30rckmI68Cyf57AhLN7eh1JQkyFLuJzgYDj1//7D5768z8YlZbMwlvGcla3Tl7HkjBQoYv42JGaOu5fuZn1W/dx3Zg0Hr1G6+V+pkIX8anCg1V894UN7NpfycPfGs5tFw7QernPqdBFfKjiaD13/H4DJeU1LL1jAhdkar08GqjQRXwmEHDMX5lLfkkFi29XmUcTvbmxiM8seCufdXn7eHDqMCYN7uV1HGlDKnQRH1m/dR9Prt/Btef3445JA72OI21MhS7iE/klR7hvRS6j0pJ57NqR2gCNQip0ER8oq67jzsU5JMR1YOEtY/XUxCilTVGRCNcQcNy7fBNFh6p48c6JetFQFFOhi0S4X/5pO29vL+XRa85lnN7+NqppyUUkgr2+uZhn397JzPEZ+mAKUaGLRKq84jJ++Mpmsvp355GrR3gdR9oBFbpIBDpYWcvsxTl06xTPM7eMIT5Wv8qiNXSRiFPXEGDuso2UVhzl5bsuILWLPmlIGumfdZEI8+gft/G3ggM8fs1IRqd38zqOtCMqdJEIsjK7kBfe+4TbLxrIdWPTvI4j7YwKXSRCbNp9iIde/YiLBvXkwalDvY4j7ZAKXSQClJTXMGdpDr2TO7Jg5hhiY/SrK1+mTVGRdu5ofQNzluZQXl3PqrsvpHtivNeRpJ1SoYu0Y845Hl6dx8bdh3n6pjEM69vV60jSjulxm0g7tvSD3SzfUMjcSzOZNqqv13GknVOhi7RTHxQc4JE1eVw2NJX5k8/xOo5EABW6SDu053A1dy/bSEbPzjw14zw6dNB7m0vrgip0M5tiZtvNLN/MHviKMZeYWa6Z5ZnZ/4U2pkj0qK5t4K4l2dTWB1g0K4uuCXFeR5II0eqmqJnFAE8Dk4EiYIOZrXHObW02phvwDDDFObfbzFLDFVjEz5xzPLBqC3nF5Tx3axaDUpO8jiQRJJgZ+ngg3zlX4JyrBZYD008YcxOwyjm3G8A5VxLamCLR4bl3d7E6t5j5k4dw+bDeXseRCBNMofcDCpsdFzVd1twQoLuZvW1mOWZ2a0tXZGazzSzbzLJLS0tPL7GIT72zo5TH39zG1JF9mHvpIK/jSAQKptBb2o1xJxzHAmOBacA3gH8xsyFf+ibnFjnnspxzWSkpKaccVsSvPj1QyT0vbWJI7y48cf1ofcCznJZgXlhUBKQ3O04DilsYs985VwlUmtk7wGhgR0hSivhYxdF67lycjRksmpVFYke93k9OTzAz9A3AYDMbaGbxwAxgzQljVgNfM7NYM+sMTAC2hTaqiP8EAo75K3PJL6lgwcwxZPTs7HUkiWCtTgWcc/VmNg9YB8QAzzvn8sxsTtP5hc65bWb238AWIAA855z7KJzBRfxgwVv5rMvbx0PThjFpcC+v40iEC+qxnXNuLbD2hMsWnnD8BPBE6KKJ+Nv6rft4cv0Orj2/H3dMGuh1HPEBvVJUxAP5JUe4b0Uuo9KSeezakdoElZBQoYu0sbLqOu5cnENCXAcW3jKWhLgYryOJT2g7XaQNNQQc9y7fRNGhKl68cyJndevkdSTxERW6SBv65Z+28/b2Uh695lzGDejhdRzxGS25iLSR1zcX8+zbO5k5PoObJ/T3Oo74kApdpA3kFZfxw1c2k9W/O49cPcLrOOJTKnSRMDtYWcvsxTl06xTPM7eMIT5Wv3YSHlpDFwmjuoYAc5dtpLTiKK/MuYDULgleRxIf01RBJIwe/eM2/lZwgMevGcmotG5exxGfU6GLhMnK7EJeeO8Tbr9oINeNTfM6jkQBFbpIGGzafYiHXv2Iiwb15MGpQ72OI1FChS4SYiXlNcxZmkPv5I4smDmG2Bj9mknb0KaoSAgdrW9gztIcyqvrWXX3hXRPjPc6kkQRFbpIiDjneHh1Hht3H+bpm8YwrG9XryNJlNFjQZEQWfr+pyzfUMjcSzOZNqqv13EkCqnQRULgg4IDPPL6Vi4bmsr8yed4HUeilApd5AztOVzN3cs2ktGzM0/NOI8OHfTe5uINFbrIGaiubeCuJdnU1gf47a1ZdE2I8zqSRDFtioqcJuccD6zaQl5xOc/dmkVmSpLXkSTKaYYucpqee3cXq3OLmT95CJcP6+11HBEVusjpeGdHKY+/uY2pI/sw99JBXscRAVToIqfs0wOV3PPSJob07sIT14/WBzxLu6FCFzkFFUfruXNxNmawaFYWiR21DSXthwpdJEiBgGP+ylzySypYMHMMGT07ex1J5DgqdJEgLXgrn3V5+3hw6jAmDe7ldRyRL1GhiwRh/dZ9PLl+B9ee3487Jg30Oo5Ii1ToIq3ILznCfStyGZWWzGPXjtQmqLRbQRW6mU0xs+1mlm9mD5xk3DgzazCz60MXUcQ7ZdV13Lk4h4S4Dvxm1lgS4mK8jiTylVotdDOLAZ4GrgKGAzPNbPhXjPs3YF2oQ4p4oSHguHf5JooOVfHsLWPpm9zJ60giJxXMDH08kO+cK3DO1QLLgektjLsH+ANQEsJ8Ip755Z+28/b2Un529QjGDejhdRyRVgVT6P2AwmbHRU2XHWNm/YBrgIUnuyIzm21m2WaWXVpaeqpZRdrM65uLefbtndw0IYObJ/T3Oo5IUIIp9JZ2gNwJx08BP3LONZzsipxzi5xzWc65rJSUlGAzirSpvOIyfvjKZsYN6M7PvjXC6zgiQQvmZW5FQHqz4zSg+IQxWcDypt3/XsBUM6t3zr0WkpQibeRgZS2zF+fQrVM8z9w8lvhYPRFMIkcwhb4BGGxmA4E9wAzgpuYDnHPHnphrZi8Ab6jMJdLUNQSYu2wjpRVHeWXOBaR06eh1JJFT0mqhO+fqzWwejc9eiQGed87lmdmcpvMnXTcXiRSP/nEbfys4wJM3jGZUWjev44icsqDeWcg5txZYe8JlLRa5c+62M48l0rZWZhfywnufcMekgVw7Js3rOCKnRQuEEvU27T7EQ69+xKRBvfjxVUO9jiNy2lToEtVKymuYszSH3skd+c+Z5xMbo18JiVx6M2eJWkfrG5izNIfy6npW3X0h3RPjvY4kckZU6BKVnHM8vDqPjbsP88zNYxjWt6vXkUTOmB5fSlRa+v6nLN9QyLxLBzF1ZF+v44iEhApdos4HBQd45PWtXD40lfsnD/E6jkjIqNAlquw5XM3dyzaS0bMzv5pxHh066L3NxT9U6BI1qmsbuGtJNrX1AX57axZdE+K8jiQSUtoUlajgnOOBVVvIKy7nd9/JIjMlyetIIiGnGbpEhefe3cXq3GJ+cOU5XDa0t9dxRMJChS6+986OUh5/cxtTR/bh7ksyvY4jEjYqdPG1Tw9Ucs9LmxjSuwtPXD9aH/AsvqZCF9+qOFrPnYuzMYPf3ppFYkdtGYm/6SdcfCkQcMxfmcvO0koW3z6e9B6dvY4kEnaaoYsvLXgrn3V5+3hw6jAuGtTL6zgibUKFLr6zfus+nly/g2vH9OP2iwZ4HUekzajQxVfyS45w34pcRqUl89g1I7UJKlFFhS6+sftAFf/8+2wS4mL4zayxJMTFeB1JpE1pU1R84d1/lDLvxU0APH/bOPomd/I4kUjbU6FLRHPOseidAv7tvz9mSO8uLJqVRUZPPaNFopMKXSJWVW09P/rDh7y+uZhpI/vyxLdH0TleP9ISvfTTLxGp8GAVs5fk8PFn5fxoylDmXHy2NkAl6qnQJeL8NX8/c1/cSCDg+K/bxnHJOaleRxJpF1ToEjGcc/zuL7t4bO02BqUmsWhWFgN6JXodS6TdUKFLRKiubeDHq7bwWm4xU0b04Zc3jCZJ780ichz9Rki7V3SoiruW5LB1bzk/uHIIcy8dpPVykRao0KVde2/nfua9uIm6hgC/+06WPpxC5CSCeqWomU0xs+1mlm9mD7Rw/mYz29L09Z6ZjQ59VIkmzjme/8suZv3u7/RIjGfNvEkqc5FWtDpDN7MY4GlgMlAEbDCzNc65rc2G7QIuds4dMrOrgEXAhHAEFv+rqWvgwVc/ZNXGPVw5vDdP3nie1stFghDMb8l4IN85VwBgZsuB6cCxQnfOvdds/PtAWihDSvQoPlzNXUty+HBPGfdPHsK8SwfRoYPWy0WCEUyh9wMKmx0XcfLZ9x3Amy2dMLPZwGyAjIyMICNKtHi/4ABzl22ktj7Ac7dmccVwLbGInIpgCr2l6ZFrcaDZpTQW+qSWzjvnFtG4HENWVlaL1yHRxznH4r99ys/f2EpGz8789tYsMlOSvI4lEnGCKfQiIL3ZcRpQfOIgMxsFPAdc5Zw7EJp44nc1dQ089NpHvJJTxBXDUnnyxvPomhDndSyRiBRMoW8ABpvZQGAPMAO4qfkAM8sAVgGznHM7Qp5SfGlvWTVzluSwuaiMey8fzL2XD9Z6ucgZaLXQnXP1ZjYPWAfEAM875/LMbE7T+YXAT4GewDNNL/iod85lhS+2RLoNnxzke0tzqK5t4DezxvKNEX28jiQS8cw5b5ays7KyXHZ2tie3Ld5xzrH0g908siaP9B6dWTRrLIN7d/E6lkjEMLOcr5ow68m90maO1jfw09fyWJFdyGVDU/nVjeeR3Enr5SKhokKXNrGvvIY5S3PYtPsw91w2iPuuGKL1cpEQU6FL2OV8epA5SzdSebSehbeMYcq5fb2OJOJLKnQJqxc/2M3Daz7irG6dWHrHBM7po/VykXBRoUtYHK1v4GdrtvLS33dz8ZAUfj3jfJI7a71cJJxU6BJyJU3r5Rt3H+Z7l2TygyvPIUbr5SJhp0KXkNq4+xBzluRwpKaep28aw7RRWi8XaSsqdAmZ5X/fzU9X59EnOYHFd4xnaJ+uXkcSiSoqdDkjzjk+2lPO83/dxaub9vC1wb34z5nn061zvNfRRKKOCl1Oy6HKWl7L3cPK7CK27S2nY2wH7r4kk/laLxfxjApdgtYQcPw1fz8rsgtZn7eP2oYAo9KS+fk/ncvVo8/Sqz5FPKZCl1YVHqzi5ZwiXskupLishm6d47h5YgbfHpvO8LO0Ti7SXqjQpUU1dQ2sy/uMFRsKeW/nAczga4NT+Mm04VwxPJWOsTFeRxSRE6jQ5ZjPNzhXZheyOncP5TX1pPfoxP2Th3D92DTO6tbJ64gichIqdGlxg/Oqc/tww7h0Jg7sqTfREokQKvQopQ1OEf9RoUcZbXCK+JcKPQq0tMH5dW1wiviOCt2nvmqDc/7kIVynDU4RX1Kh+4w2OEWilwrdB7TBKSKgQo84zjn2ltWws7SC/JLGr7c+LqG4rIbuTRucN2SlM6yvNjhFoo0KvZ2qrQ/wyYFKdjaV9s7SCnaWVrKztIKq2oZj47omxDKmf3dtcIqICt1rZdV1zQq7gp0ljcW9+2AVDQF3bFy/bp04OyWRG8elk5mSxKDUJDJTkuiVFI+Z1sVFRIXeJgIBx97ymhNm2xXkl1Syv+LosXHxMR0Y2CuRYX278K1RfclsKu2zUxLpHK//VSJycmqJEDpa38An+6uOzbTzm4q7oLTyuGWS5E5xDEpN4rKhKcdm2pkpSaT36Kz3EheR06ZCD4JzjqraBo7U1FNeU8eRmjrKq+spPXK02Wy7gt0Hq2i2SkK/bp0YlJrE+AE9yUxNZFBKEpmpSfRM1DKJiIReVBR6XUOAIzX1x4r4SE0d5TV1lNfUU15d16yoWzhu+rP5enZz8bEdOLtXIiPOSubq8/qRmZLIoNQkzu6VRKd4bVCKSNsJqtDNbArwH0AM8Jxz7hcnnLem81OBKuA259zGEGcFoORIDXl7ylsp5MZznxd4dV1Dq9fbpWMsXTvF0SUhlq4JcfTpmsCQ3l8cd0k4/nyXhFh6JnakX/dOWiYRkXah1UI3sxjgaWAyUARsMLM1zrmtzYZdBQxu+poAPNv0Z8j9fddB5r246bjL4mM60LVTLF0S4uia0Phnn+SEL4r4uEL+Yszn35PUMValLCIRL5gZ+ngg3zlXAGBmy4HpQPNCnw4sds454H0z62ZmfZ1ze0Md+MLMXrx694XHzZYT4rS0ISISTKH3AwqbHRfx5dl3S2P6AccVupnNBmYDZGRknGpWAHokxtMjMf60vldExM86BDGmpbWIE3cIgxmDc26Rcy7LOZeVkpISTD4REQlSMIVeBKQ3O04Dik9jjIiIhFEwhb4BGGxmA80sHpgBrDlhzBrgVms0ESgLx/q5iIh8tVbX0J1z9WY2D1hH49MWn3fO5ZnZnKbzC4G1ND5lMZ/Gpy1+N3yRRUSkJUE9D905t5bG0m5+2cJmf3fA3NBGExGRUxHMkouIiEQAFbqIiE+o0EVEfEKFLiLiEyp0ERGfUKGLiPiECl1ExCdU6CIiPqFCFxHxCRW6iIhPqNBFRHxChS4i4hPW+L5aHtywWSnwqSc3Hjq9gP1eh2hHdH8cT/fHF3RfHO9M7o/+zrkWPyHIs0L3AzPLds5leZ2jvdD9cTzdH1/QfXG8cN0fWnIREfEJFbqIiE+o0M/MIq8DtDO6P46n++MLui+OF5b7Q2voIiI+oRm6iIhPqNBFRHxChR4iZvYDM3Nm1svrLF4ysyfM7GMz22Jmr5pZN68ztTUzm2Jm280s38we8DqPl8ws3czeMrNtZpZnZvd6nclrZhZjZpvM7I1QX7cKPQTMLB2YDOz2Oks7sB441zk3CtgB/NjjPG3KzGKAp4GrgOHATDMb7m0qT9UD851zw4CJwNwovz8A7gW2heOKVeih8Svg/wFRv8PsnPuTc66+6fB9IM3LPB4YD+Q75wqcc7XAcmC6x5k845zb65zb2PT3IzQWWT9vU3nHzNKAacBz4bh+FfoZMrOrgT3Ouc1eZ2mHbgfe9DpEG+sHFDY7LiKKC6w5MxsAnA984G0STz1F4+QvEI4rjw3HlfqNmf0Z6NPCqZ8ADwJXtm0ib53s/nDOrW4a8xMaH24va8ts7YC1cFnUP3IzsyTgD8D3nXPlXufxgpl9EyhxzuWY2SXhuA0VehCcc1e0dLmZjQQGApvNDBqXFzaa2Xjn3GdtGLFNfdX98Tkz+w7wTeByF30vdCgC0psdpwHFHmVpF8wsjsYyX+acW+V1Hg9dBFxtZlOBBKCrmS11zt0SqhvQC4tCyMw+AbKcc1H7rnJmNgV4ErjYOVfqdZ62ZmaxNG4GXw7sATYANznn8jwN5hFrnOn8HjjonPu+13nai6YZ+g+cc98M5fVqDV1CbQHQBVhvZrlmttDrQG2paUN4HrCOxg3AldFa5k0uAmYBlzX9POQ2zVAlDDRDFxHxCc3QRUR8QoUuIuITKnQREZ9QoYuI+IQKXUTEJ1ToIiI+oUIXEfGJ/w9+Se0yNI6QRAAAAABJRU5ErkJggg==\n"
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def sigmoid(x):\n",
    "    return 1/(1+np.exp(-x))\n",
    "\n",
    "# 绘制sigmoid函数\n",
    "x = np.arange(-5.0,5.0,1)\n",
    "y = sigmoid(x)\n",
    "plt.plot(x,y)\n",
    "plt.ylim(-0.1,1.1)# 指定y轴范围\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-25T07:46:46.570348500Z",
     "start_time": "2023-09-25T07:46:46.328351800Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 3.2.6.非线性函数\n",
    "<p>事实上,神经网络的激活函数必须使用非线性函数,否则神经网络的多层结构毫无意义</p>\n",
    "\n",
    "### 3.2.7.ReLU函数\n",
    "<p>ReLU函数在输入大于0时,直接输出该值,在输入小于等于0时，输出0</p>"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "outputs": [
    {
     "data": {
      "text/plain": "<Figure size 432x288 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAD4CAYAAADFAawfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAWtUlEQVR4nO3deXiU9bnG8fsxgMgmKhGRxaAissgSIlBRW5cq7qeLlkXPabWlh8Widak92tq9vVq3VsAeTrW1JYC41dZqFRdqXWsWQDCA7DsJsq/ZnvNHEho1mElm3nnfmfl+rovLQIaZZyDe8+PJzNzm7gIARNcRYQ8AAPh0BDUARBxBDQARR1ADQMQR1AAQcS2CuNJOnTp5Tk5OEFcNAGmpsLBwq7tnN/S5QII6JydHBQUFQVw1AKQlM1tzuM+x+gCAiIvpRG1mqyXtllQlqdLd84IcCgDwb01ZfZzn7lsDmwQA0CBWHwAQcbEGtUt60cwKzWxcQxcws3FmVmBmBWVlZYmbEAAyXKxBPcLdcyVdImmimZ378Qu4+3R3z3P3vOzsBp9hAgBohpiC2t031v63VNLTkoYGORQA4N8aDWoza2tm7es+lnSRpEVBDwYAqeRfq7bp4ddXKYi3jo7lWR+dJT1tZnWXn+nuf0/4JACQosp2H9SkmUVqe2QLjR7aXW1aJfa1hI1em7uvlDQwobcKAGmiqto1eXaxdu6v0KPXD014SEsBvYQcADLFAy8t05srPtQvvzxAfbp0COQ2eB41ADTTvKWlevCV5bp6SDddk9c9sNshqAGgGTbu2K+bH5uv009orx9d1T/Q2yKoAaCJyiurNXFmkSqqXNPG5uqoVlmB3h47agBool88v0TFa3do6phcnZzdLvDb40QNAE3w/Hub9Mgbq/TVs3J02YAuSblNghoAYrRq617d9sRCDezeUf9zaZ+k3S5BDQAxOFBRpfEzCpV1hGnqmMFq1SJ58cmOGgBi8IO/LNaSzbv1yFfz1O2YNkm9bU7UANCIJwvXa/a76zThc6fo/NM7J/32CWoA+BRLNu/SnX9+T8N6Hqtvf/60UGYgqAHgMPYcrNSE/CK1b91SD44ZrBZZ4UQmQQ0ADXB33fHkQq3eule/GTVYx7dvHdosBDUANOBPb6/Rsws36ZaLeuszpxwX6iwENQB8zIJ1O/TjZ9/Xeb2zNf6zp4Q9DkENAPXt2FeuCflFOr59a913zSAdcYSFPRLPowaAOtXVrlvmLFDp7gN6/L/P0jFtW4U9kiRO1ABwyP++tlIvLynVnZf20aDuHcMe5xCCGgAkvb3yQ93z4lJddkYX/ddZOWGP8xEENYCMV7r7gG6cVayTjm2jX3zpDNWWeUcGO2oAGa2q2jV51nztPlChP90wVO1btwx7pE8gqAFktPvnLtNbKz/Ur748QKefEEw5bbxYfQDIWK8uLdWUV5frmrxuujrActp4EdQAMtKGJJbTxougBpBxyiurNTG/SJVVroeuHaLWLYMtp40XO2oAGefnz5do/rodmjY2Vz07tQ17nEZxogaQUZ57b5N+/8ZqffWsHF16RnLKaeNFUAPIGKu27tXtTyzUoCSX08aLoAaQEerKaVtkmaaOzU1qOW282FEDyAh3P1NTTvv7r52prh2PCnucJkmdhxQAaKYnCtfrsYJ1mnjeKTqv9/Fhj9NkMQe1mWWZWbGZPRvkQACQSEs279Jdf35Pw08+VjdfGE45bbyacqKeLKkkqEEAINHql9P+ZnR45bTximlqM+sm6TJJvwt2HABIjPrltA+ODrecNl6xPrw8IOl2SdUBzgIACVNXTnvbxadr+MnhltPGq9GgNrPLJZW6e2EjlxtnZgVmVlBWVpawAQGgqebXltNecPrx+ua5J4c9TtxiOVGPkHSlma2WNFvS+WY24+MXcvfp7p7n7nnZ2dkJHhMAYrNjX7km1pbT3nvNwEiU08ar0aB29++6ezd3z5E0StIr7n5t4JMBQBNVV7u+PWeBynYf1LSxuerYJhrltPFKzW+BAkADfvvaCr2ypFR3Xd5HAyNUThuvJr0y0d3nSZoXyCQAEIe3V36oe15YqssHdNF1w08Ke5yE4kQNIOXVldPmdGqrX3xpQOTKaePFe30ASGmVVdWHymln3DBM7Y5Mv1hLv3sEIKPc/1JNOe09Vw9U7xPahz1OIFh9AEhZry4p1dRXV+gred315SHdwh4nMAQ1gJS0Ycd+3Txnvvp06aAfXtUv7HECRVADSDn1y2mnjc2NfDltvNhRA0g5P3uuppz2oRQpp40XJ2oAKeVvCzfpD2+u1tdG5OiSFCmnjRdBDSBlrCzbo+88uVCDe3TUdy9JnXLaeBHUAFLCgYoqTcgvUsss09QxqVVOGy921ABSwvefWaSlW3br9189UyemWDltvDLnIQlAynq8YJ3mFKzXpPNO1edSsJw2XgQ1gEhbsnmXvvfMIp11ynG6KUXLaeNFUAOIrN0HKjR+RpE6tG6pX48arKw0KAFoDnbUACLJ3XXHU+9p7bZ9mvn1Ycpuf2TYI4WGEzWASHr0zdX628JNuvWi3hqW4uW08SKoAURO8drt+ulzJWlTThsvghpApGzfW65JM4vTqpw2XuyoAURGTTntfJXtPqjH//szaVNOGy9O1AAi46F/rNCrS8vSrpw2XgQ1gEh4a8WHuvfFpbpi4IlpV04bL4IaQOjql9P+/ItnpF05bbzYUQMIVWVVtb41q1h7DlYo/+vpWU4bL/5EAITq/peW6e2V23RvGpfTxovVB4DQ1JXTjjqzu76UxuW08SKoAYSirpy2b5cO+sGV6V1OGy+CGkDS1ZXTVmVIOW282FEDSLr65bQ5GVBOGy9O1ACSqq6c9voRPTOmnDZeBDWApKkrp83t0VF3XHJ62OOkDIIaQFLsL/93Oe2UDCunjRc7agBJkcnltPFq9CHNzFqb2b/MbIGZLTazHyZjMADpY8676/R44XrdmKHltPGK5UR9UNL57r7HzFpKet3Mnnf3twOeDUAaKNlUU0474tTjNDlDy2nj1WhQu7tL2lP705a1PzzIoQCkh90HKjQhv0hHH9VSD3wlc8tp4xXTNt/MssxsvqRSSXPd/Z0GLjPOzArMrKCsrCzRcwJIMe6u7zy5UGu37dOUMbkZXU4br5iC2t2r3H2QpG6ShppZ/wYuM93d89w9Lzs7O9FzAkgxf3hztZ57b7Nuu7i3hvY8NuxxUlqTnh/j7jskzZM0MpBpAKSF4rXb9bPnSnRhn+M17hzKaeMVy7M+ss2sY+3HR0m6UNKSoAcDkJrqymk7d2ite68eRDltAsTyrI8ukh41syzVBPscd3822LEApKLqatfNteW0T4z/jI5u0zLskdJCLM/6WChpcBJmAZDiHvrHCs1bWqYfX9VPA7pRTpsovIYTQEK8uWLroXLaaymnTSiCGkDcSncd0LdmzaecNiC81weAuFRWVetGymkDxZ8ogLjcN3eZ3lm1TfddQzltUFh9AGi2V5Zs0bR5KzR6aHd9MZdy2qAQ1ACaZf32fbr5sQXq26WD7r6CctogEdQAmuxgZZUm5hepuppy2mRgRw2gyX72txItWL9Tv72Wctpk4EQNoEn+umCjHn1rjW44u6dG9qecNhkIagAxW1G2R3dQTpt0BDWAmOwvr9KEGUVq1eIITRmTq5ZZxEeysKMGEJPvPbNIy0p369GvDaWcNsl4SATQqDnvrtMThet14/m9dO5pFIMkG0EN4FO9v7GmnPbsUztp8gW9wh4nIxHUAA5r94EKTZxZpI5tWuqBUYMopw0JO2oADapfTjt73HB1akc5bVg4UQNo0O/fqCmnvf3i3jozh3LaMBHUAD6haO12/fz5El3Yp7PGnUs5bdgIagAfsX1vuSblF9WW0w6kBCAC2FEDOKSunHbrnnI9Of4symkjghM1gEOmzVuueUvL9L0r+uqMbkeHPQ5qEdQAJNWU0943d5muHHiirh3WI+xxUA9BDeBQOW1PymkjiR01kOEqq6o1aVax9h6s1MxvDFNbymkjh78RIMPdO3eZ/lVbTntaZ8ppo4jVB5DBXi7ZoofmrdDooT0op40wghrIUOu27dO35yxQvxM76O4r+oY9Dj4FQQ1koIOVVZo0s0jVTjltKmBHDWSgnx4qpx2ik46jnDbqOFEDGeavCzbqj2+t0dfP7qmR/U8IexzEgKAGMkhdOe2Qk47RdyinTRmNBrWZdTezV82sxMwWm9nkZAwGILH2lVdq/IxCHdkyS1PGDKacNoXEsqOulHSLuxeZWXtJhWY2193fD3g2AAni7rrrz4v0QekePfq1oepyNOW0qaTRh1R33+TuRbUf75ZUIqlr0IMBSJzH3l2np4o2UE6bopr0bx8zy5E0WNI7DXxunJkVmFlBWVlZYqYDELfFG3fq+39ZTDltCos5qM2snaQnJd3k7rs+/nl3n+7uee6el53NIzYQBbsOVGhCfpGOoZw2pcX0PGoza6makM5396eCHQlAIri7bn98odZv3085bYqL5VkfJulhSSXufl/wIwFIhEfeWK2/L96s74yknDbVxbL6GCHpOknnm9n82h+XBjwXgDgUrtmunz9Xos/37axvnEM5baprdPXh7q9LYrEFpIhte8s1aWaRunRsrXsop00LvNcHkEaqq103PTZfH9aV0x5FOW064KVJQBqZ+upyvbasTN+nnDatENRAmnhz+Vbd/9IyXTXoRI2lnDatENRAGtiy64C+NbtYJ2e308++QDltumFHDaS4yqpq3TirWHsPVmnWN3Ipp01D/I0CKe6eF2vKae//ykD1opw2LbH6AFLYyyVb9Nt/1JTTfmEw5bTpiqAGUtS6bft082Pz1b8r5bTpjqAGUtDByipNnFkklzRtzBDKadMcO2ogBf3k2RItXL9T068boh7HtQl7HASMEzWQYv6yYKP+9PYafeOcnrqoH+W0mYCgBlLI8tKactq8k47R7SMpp80UBDWQIvaVV2pCfqFat8zSg5TTZhR21EAKqF9O+8frKafNNDwkAymgrpx28gW9dE4vqu4yDUENRFxdOe05vTrpxvMpp81EBDUQYXXltMe2aaUHvkI5baZiRw1EVP1y2sfGDddxlNNmLE7UQEQ9/Poq/X3xZt0x8nTlUU6b0QhqIIIK12zTL55foov6dtbXz+kZ9jgIGUENRExNOW2xTux4lH5FOS3EjhqIlEPltHvL9RTltKjFiRqIkCm15bQ/uKKf+nelnBY1CGogIt6oLaf9wuCuGj20e9jjIEIIaiACtuw6oMmzi3Vqdjv99Av92UvjI9hRAyGrqKrWpJlF2ldepdnjctWmFf9b4qP4igBCds8LS/Xu6u369ahBOvV4ymnxSaw+gBDNfX+L/ve1lRo7rIeuGtQ17HEQUQQ1EJJ12/bpljk15bTfu5xyWhweQQ2E4EBFlSbkU06L2LCjBkLwk7+9r/c2UE6L2DR6ojazR8ys1MwWJWMgIN09M3+DZry9VuPOPZlyWsQkltXHHySNDHgOICMsL92t7z71ns7MOUa3Xdw77HGQIhoNand/TdK2JMwCpLV95ZUaP6NIR7XM0oOjcymnRczYUQNJ4O666+lFWl62R3+6fphOOLp12CMhhSTsId3MxplZgZkVlJWVJepqgbQw+911eqq4ppz27F6dwh4HKSZhQe3u0909z93zsrNpSQbqLNqwU3fXltN+i3JaNANLMiBAuw5UaOLMf5fTHkE5LZohlqfnzZL0lqTeZrbezG4Ifiwg9bm7bnt8gTZs36+pYwdTTotma/Sbie4+OhmDAOnm4ddX6YXFW3TXZX005CTKadF8rD6AANSV017cr7NuOJtyWsSHoAYS7MM9BzUxv1hdjzlKv/wy5bSIH8+jBhKoqracdtu+cj09gXJaJAYnaiCBpryyXP/8YKt+eGU/9TuRclokBkENJMjrH2zVAy8v0xcHd9WoMymnReIQ1EACbN5ZU07b6/h2+gnltEgwghqIU0VVtW6cVaT9FVWaNpZyWiQeX1FAnCinRdA4UQNxqCunvXY45bQIDkENNFNdOe0ZXY+mnBaBIqiBZqgrp5WkaWNzdWQLymkRHHbUQDPUldP+33/mqfuxlNMiWJyogSaqK6f95rkn6/N9O4c9DjIAQQ00Qf1y2lspp0WSENRAjOrKadu0ytKUMZTTInnYUQMxcHfdWVtOO+OGYercgXJaJA9HAiAGs/61Tk8Xb9DNF56mEadSTovkIqiBRizasFM/+OtinXtatiadd2rY4yADEdTAp9i5v0IT8ot0XFvKaREedtTAYdSV027csV+PfXO4jm3bKuyRkKE4UQOH8fDrq/Ti+1t0xyWnU06LUBHUQAPqymlH9juBclqEjqAGPuYj5bRXD6AEAKFjRw3UU7+c9qnxZ6lDa8ppET5O1EA9D77ywaFy2v5dKadFNBDUQK1/flCmX7/8gb6YSzktooWgBiRt2rlfN82eX1NO+x+U0yJaCGpkvIqqak2aWVxbTjuEclpEDl+RyHi//PsSFa7Zrt+MHqxTj28X9jjAJ3CiRkZ7YfFm/d8/V+m64SfpyoEnhj0O0CCCGhlr7Yf7dOvjCzSg29G66/I+YY8DHBZBjYx0oKJK4/MLZZKmjqGcFtEWU1Cb2UgzW2pmy83sjqCHAoL2o2ff1+KNu3TfNYMop0XkNRrUZpYlaaqkSyT1lTTazPoGPRgQlD8Xb9DMd9bqm589WRdSTosUEMuzPoZKWu7uKyXJzGZLukrS+4ke5ooHX9eBiqpEXy3wEWu27dPQnGN120WU0yI1xBLUXSWtq/fz9ZKGffxCZjZO0jhJ6tGjR7OGOSW7rcqrqpv1e4FY5fY4RrdcdJpaUE6LFBFLUDf0Ei3/xC+4T5c0XZLy8vI+8flYPDBqcHN+GwCktViOFOsl1X/jg26SNgYzDgDg42IJ6ncl9TKznmbWStIoSX8JdiwAQJ1GVx/uXmlmkyS9IClL0iPuvjjwyQAAkmJ8rw93f07ScwHPAgBoAN/2BoCII6gBIOIIagCIOIIaACLO3Jv12pRPv1KzMklrEn7FweskaWvYQyRZJt5nKTPvN/c52k5y9+yGPhFIUKcqMytw97yw50imTLzPUmbeb+5z6mL1AQARR1ADQMQR1B81PewBQpCJ91nKzPvNfU5R7KgBIOI4UQNAxBHUABBxBHUDzOxWM3Mz6xT2LMlgZr8ysyVmttDMnjazjmHPFJRMLGo2s+5m9qqZlZjZYjObHPZMyWJmWWZWbGbPhj1LPAjqjzGz7pI+L2lt2LMk0VxJ/d19gKRlkr4b8jyByOCi5kpJt7h7H0nDJU3MkPstSZMllYQ9RLwI6k+6X9LtaqBuLF25+4vuXln707dV0+KTjg4VNbt7uaS6oua05u6b3L2o9uPdqgmuruFOFTwz6ybpMkm/C3uWeBHU9ZjZlZI2uPuCsGcJ0fWSng97iIA0VNSc9oFVn5nlSBos6Z1wJ0mKB1Rz6Er5xuyYigPSiZm9JOmEBj51p6T/kXRRcidKjk+73+7+TO1l7lTNP5PzkzlbEsVU1JyuzKydpCcl3eTuu8KeJ0hmdrmkUncvNLPPhT1PvDIuqN39woZ+3czOkNRT0gIzk2r++V9kZkPdfXMSRwzE4e53HTP7L0mXS7rA0/fJ9Rlb1GxmLVUT0vnu/lTY8yTBCElXmtmlklpL6mBmM9z92pDnahZe8HIYZrZaUp67p8o7bzWbmY2UdJ+kz7p7WdjzBMXMWqjmm6UXSNqgmuLmMeneAWo1J49HJW1z95vCnifZak/Ut7r75WHP0lzsqCFJUyS1lzTXzOab2W/DHigItd8wrStqLpE0J91DutYISddJOr/273d+7UkTKYITNQBEHCdqAIg4ghoAIo6gBoCII6gBIOIIagCIOIIaACKOoAaAiPt//J064L81sDIAAAAASUVORK5CYII=\n"
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 定义RELU函数\n",
    "def relu(x):\n",
    "    return np.maximum(0,x)\n",
    "\n",
    "x = np.arange(-5,5,0.1)\n",
    "y = relu(x)\n",
    "plt.plot(x,y)\n",
    "plt.show()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-25T07:59:11.633799400Z",
     "start_time": "2023-09-25T07:59:11.376806200Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 3.3 多维数组的运算\n",
    "### 3.3.1.多维数组"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 2 3 4]\n",
      "1\n",
      "(4,)\n",
      "4\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "A = np.array([1,2,3,4])\n",
    "print(A)\n",
    "print(np.ndim(A)) # 获取数组的维数\n",
    "print(A.shape) # 获取数组的形状\n",
    "print(A.shape[0]) # 数组第一维的大小"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-25T08:05:46.064970700Z",
     "start_time": "2023-09-25T08:05:46.035968200Z"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1 2]\n",
      " [3 4]\n",
      " [5 6]]\n",
      "2\n",
      "(3, 2)\n"
     ]
    }
   ],
   "source": [
    "# 二维数组\n",
    "B = np.array([[1,2],[3,4],[5,6]])\n",
    "print(B)\n",
    "print(np.ndim(B))\n",
    "print(B.shape)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-25T08:07:16.644972400Z",
     "start_time": "2023-09-25T08:07:16.620908400Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 3.3.2.矩阵乘法"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2, 2)\n",
      "(2, 2)\n"
     ]
    },
    {
     "data": {
      "text/plain": "array([[19, 22],\n       [43, 50]])"
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.array([[1,2],[3,4]])\n",
    "print(A.shape)\n",
    "B = np.array([[5,6],[7,8]])\n",
    "print(B.shape)\n",
    "np.dot(A,B)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-25T11:33:49.015938600Z",
     "start_time": "2023-09-25T11:33:48.949598400Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 3.3.3.神经网络的内积"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2,)\n",
      "[[1 3 5]\n",
      " [2 4 6]]\n",
      "(2, 3)\n",
      "[ 5 11 17]\n"
     ]
    }
   ],
   "source": [
    "X =np.array([1,2])\n",
    "print(X.shape)\n",
    "W = np.array([[1,3,5],[2,4,6]])\n",
    "print(W)\n",
    "print(W.shape)\n",
    "Y = np.dot(X,W)\n",
    "print(Y)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-25T11:40:29.312894900Z",
     "start_time": "2023-09-25T11:40:29.278675600Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 3.4.实现3层神经网络"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2, 3)\n",
      "(2,)\n",
      "(3,)\n",
      "[0.3 0.7 1.1]\n",
      "[0.57444252 0.66818777 0.75026011]\n"
     ]
    }
   ],
   "source": [
    "X = np.array([1.0,0.5]) # x1与x2\n",
    "W1 = np.array([[0.1,0.3,0.5],[0.2,0.4,0.6]]) # 权重矩阵\n",
    "B1 = np.array([0.1,.2,0.3]) #偏置矩阵\n",
    "\n",
    "print(W1.shape)\n",
    "print(X.shape)\n",
    "print(B1.shape)\n",
    "A1 = np.dot(X,W1)+B1\n",
    "\n",
    "Z1 = sigmoid(A1) # 第一层的输出\n",
    "print(A1)\n",
    "print(Z1)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-25T11:56:39.217474300Z",
     "start_time": "2023-09-25T11:56:39.118512400Z"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(3,)\n",
      "(3, 2)\n",
      "(2,)\n",
      "[0.51615984 1.21402696]\n",
      "[0.62624937 0.7710107 ]\n"
     ]
    }
   ],
   "source": [
    "# 实现第一层到第二层的信号传递\n",
    "W2 = np.array([[0.1,0.4],[0.2,0.5],[0.3,0.6]])\n",
    "B2 = np.array([0.1,0.2])\n",
    "\n",
    "print(Z1.shape)\n",
    "print(W2.shape)\n",
    "print(B2.shape)\n",
    "\n",
    "A2 = np.dot(Z1,W2)+B2\n",
    "Z2 = sigmoid(A2)\n",
    "\n",
    "print(A2)\n",
    "print(Z2)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-25T12:03:54.187108800Z",
     "start_time": "2023-09-25T12:03:54.175107400Z"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.31682708 0.69627909]\n",
      "(2,)\n"
     ]
    }
   ],
   "source": [
    "def identity_function(x):\n",
    "    return x\n",
    "\n",
    "W3 = np.array([[0.1,0.3],[0.2,0.4]])\n",
    "B3 = np.array([0.1,0.2])\n",
    "\n",
    "A3 = np.dot(Z2,W3)+B3\n",
    "Y = identity_function(A3)\n",
    "print(Y)\n",
    "print(Y.shape)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-25T12:09:33.922840800Z",
     "start_time": "2023-09-25T12:09:33.895817400Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 3.5.输出层的设计\n",
    "<p>一般而言,回归问题使用恒等函数,分类问题使用softmax函数</p>"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 3.5.1实现softmax函数"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 1.34985881 18.17414537 54.59815003]\n",
      "74.1221542101633\n",
      "[0.01821127 0.24519181 0.73659691]\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "a = np.array([0.3,2.9,4.0])\n",
    "exp_a =np.exp(a) # 指数函数\n",
    "print(exp_a)\n",
    "sum_exp_a = np.sum(exp_a) # 指数函数的和\n",
    "print(sum_exp_a)\n",
    "y = exp_a/sum_exp_a\n",
    "print(y)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-26T06:23:54.894665400Z",
     "start_time": "2023-09-26T06:23:54.763688700Z"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "outputs": [],
   "source": [
    "def softmax(a):\n",
    "    exp_a = np.exp(a)\n",
    "    sum_exp_a = np.sum(exp_a)\n",
    "    y = exp_a / sum_exp_a\n",
    "\n",
    "    return y"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-26T06:26:42.317282500Z",
     "start_time": "2023-09-26T06:26:42.295284100Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    ">- 注意,上述softmax函数有一个缺点:当a中的元素非常大时,将会导致数据溢出(因为指数运算的数量级极大),解决方法是让指数函数减去a中的最大值"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[nan nan nan]\n",
      "[  0 -10 -20]\n",
      "[9.99954600e-01 4.53978686e-05 2.06106005e-09]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "<ipython-input-4-d952f57423a9>:2: RuntimeWarning: overflow encountered in exp\n",
      "  print(np.exp(a)/np.sum(np.exp(a))) #soft函数运算,会因为溢出导致无法正确运算\n",
      "<ipython-input-4-d952f57423a9>:2: RuntimeWarning: invalid value encountered in true_divide\n",
      "  print(np.exp(a)/np.sum(np.exp(a))) #soft函数运算,会因为溢出导致无法正确运算\n"
     ]
    }
   ],
   "source": [
    "a = np.array([1010,1000,990])\n",
    "print(np.exp(a)/np.sum(np.exp(a))) #softmax函数运算,会因为溢出导致无法正确运算\n",
    "c = np.max(a) #1010\n",
    "print(a-c)\n",
    "b =np.exp(a-c) / np.sum(np.exp(a-c))\n",
    "print(b)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-26T06:37:47.109813700Z",
     "start_time": "2023-09-26T06:37:47.094772200Z"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "outputs": [],
   "source": [
    "# 因此,我们可以如下实现softmax函数\n",
    "def softmax(a):\n",
    "    c = np.max(a)\n",
    "    exp_a = np.exp(a-c) # 溢出对策\n",
    "    sum_exp_a = np.sum(exp_a)\n",
    "    y = exp_a / sum_exp_a\n",
    "\n",
    "    return y"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-26T06:40:14.865189500Z",
     "start_time": "2023-09-26T06:40:14.839066700Z"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.01650685 0.24561748 0.73787568]\n",
      "1.0\n"
     ]
    }
   ],
   "source": [
    "a = np.array([0.2,2.9,4.0])\n",
    "y = softmax(a)\n",
    "print(y)\n",
    "print(np.sum(y))"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-26T06:46:25.342570400Z",
     "start_time": "2023-09-26T06:46:25.321591800Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## 3.6.识别手写数字集\n",
    "### 3.6.1.MNIST数据集"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading train-images-idx3-ubyte.gz ... \n",
      "Done\n",
      "Downloading train-labels-idx1-ubyte.gz ... \n",
      "Done\n",
      "Downloading t10k-images-idx3-ubyte.gz ... \n",
      "Done\n",
      "Downloading t10k-labels-idx1-ubyte.gz ... \n",
      "Done\n",
      "Converting train-images-idx3-ubyte.gz to NumPy Array ...\n",
      "Done\n",
      "Converting train-labels-idx1-ubyte.gz to NumPy Array ...\n",
      "Done\n",
      "Converting t10k-images-idx3-ubyte.gz to NumPy Array ...\n",
      "Done\n",
      "Converting t10k-labels-idx1-ubyte.gz to NumPy Array ...\n",
      "Done\n",
      "Creating pickle file ...\n",
      "Done!\n",
      "(60000,)\n",
      "(10000, 784)\n",
      "(10000,)\n"
     ]
    }
   ],
   "source": [
    "from mnist import load_mnist\n",
    "(X_train,t_train),(X_test,t_test) = load_mnist(flatten=True,normalize=False)\n",
    "# 输出各个数据的形状\n",
    "print(t_train.shape)\n",
    "print(X_test.shape)\n",
    "print(t_test.shape)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-26T07:06:05.728664Z",
     "start_time": "2023-09-26T07:05:59.588030900Z"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5\n",
      "(784,)\n",
      "(28, 28)\n"
     ]
    }
   ],
   "source": [
    "# 显示MNIST图像\n",
    "from PIL import Image\n",
    "def img_show(img):\n",
    "    pil_img = Image.fromarray(np.uint8(img))\n",
    "    pil_img.show()\n",
    "\n",
    "(X_train,t_train),(X_test,t_test) = load_mnist(flatten=True,normalize=False)\n",
    "img = X_train[0]\n",
    "label = t_train[0]\n",
    "print(label)\n",
    "\n",
    "print(img.shape)\n",
    "img = img.reshape(28,28)\n",
    "print(img.shape)\n",
    "\n",
    "img_show(img)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-26T07:17:30.540264200Z",
     "start_time": "2023-09-26T07:16:48.511120200Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 3.6.2.神经网络的推理处理\n",
    "<p>现在,我们对这个MNIST数据集实现神经网络的推理处理。权重参数已经学得</p>"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "outputs": [],
   "source": [
    "import pickle\n",
    "def sigmoid(a):\n",
    "    y = 1/(1+np.exp(-a))\n",
    "    return y\n",
    "\n",
    "def get_data():\n",
    "    (X_train,t_train),(X_test,t_test) = load_mnist(normalize=True,flatten=True,one_hot_label=False)\n",
    "    return X_test,t_test\n",
    "\n",
    "def init_network():\n",
    "    with open(\"sample_weight.pkl\",'rb') as f:\n",
    "        network = pickle.load(f)\n",
    "\n",
    "    return network\n",
    "\n",
    "def predict(network,x):\n",
    "    W1,W2,W3 = network['W1'],network['W2'],network['W3']\n",
    "    b1,b2,b3 = network['b1'],network['b2'],network['b3']\n",
    "    a1 = np.dot(x,W1)+b1\n",
    "    z1 = sigmoid(a1)\n",
    "    a2 = np.dot(z1,W2)+b2\n",
    "    z2 = sigmoid(a2)\n",
    "    a3 = np.dot(z2,W3)+b3\n",
    "    y = softmax(a3)\n",
    "\n",
    "    return y"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-26T07:37:46.008900700Z",
     "start_time": "2023-09-26T07:37:45.967438700Z"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy:0.9352\n"
     ]
    }
   ],
   "source": [
    "x,t = get_data()\n",
    "network = init_network()\n",
    "\n",
    "accuracy_cnt = 0\n",
    "for i in range(len(x)):\n",
    "    y=predict(network,x[i])\n",
    "    p=np.argmax(y)# 获取频率最高的元素的索引\n",
    "    if p==t[i]:\n",
    "        accuracy_cnt+=1\n",
    "\n",
    "print('Accuracy:'+str(float(accuracy_cnt)/len(x)))"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-26T07:43:30.014584200Z",
     "start_time": "2023-09-26T07:43:27.765248600Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "### 3.6.3.批处理\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10000, 784)\n",
      "(784,)\n",
      "(784, 50)\n",
      "(50, 100)\n",
      "(100, 10)\n"
     ]
    }
   ],
   "source": [
    "## 查看刚才的神经网络的各层的权重的形状\n",
    "x,_ = get_data()\n",
    "network = init_network()\n",
    "W1,W2,W3 = network['W1'],network['W2'],network['W3']\n",
    "print(x.shape)\n",
    "print(x[0].shape)\n",
    "print(W1.shape)\n",
    "print(W2.shape)\n",
    "print(W3.shape)"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-26T07:56:55.971784100Z",
     "start_time": "2023-09-26T07:56:55.774784400Z"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy:0.9352\n"
     ]
    }
   ],
   "source": [
    "# 基于批处理的代码实现\n",
    "x,t = get_data()\n",
    "network = init_network()\n",
    "\n",
    "batch_size = 100#批数量\n",
    "accuracy_cnt = 0\n",
    "for i in range(0,len(x),batch_size):\n",
    "    x_batch = x[i:i+batch_size]\n",
    "    y_batch = predict(network,x_batch)\n",
    "    p = np.argmax(y_batch,axis=1)\n",
    "    accuracy_cnt +=np.sum(p == t[i:i+batch_size])\n",
    "\n",
    "print('Accuracy:'+str(float(accuracy_cnt)/len(x)))"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2023-09-26T08:10:58.916202Z",
     "start_time": "2023-09-26T08:10:58.645205500Z"
    }
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [],
   "metadata": {
    "collapsed": false
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
